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Background: Outside the USA, Agency for Healthcare Research and Quality (AHRQ) prevention quality indicators 
(PQIs) have been used to compare the quality of primary care services only at a national or regional level. However, 
in several national health systems, primary care is not directly managed by the regions but is in charge of smaller 
territorial entities. We evaluated whether PQIs might be used to compare the performance of local providers such 
as Italian local health authorities (LHAs) and health districts. Methods: We analysed the hospital discharge 
abstracts of 44 LHAs (and 11 health districts) of five Italian regions (including ^18 million residents) in 2008-10. 
Age-standardized PQI rates were computed following AHRQ specifications. Potential predictors were investigated 
using multilevel modelling. Results: We analysed 11470722 hospitalizations. The overall rates of preventable 
hospitalizations (composite PQI 90) were 1012, 889 and 988 (x 100 000 inhabitants) in 2008, 2009 and 2010, 
respectively. Composite PQIs were able to differentiate LHAs and health districts and showed small variation in 
the performance ranking over years. Conclusion: Although further research is required, our findings support the 
use of composite PQIs to evaluate the performance of relatively small primary health care providers (50 000-60 000 
enrollees) in countries with universal health care coverage. Achieving high precision may be crucial for a structured 
quality assessment system to align hospitalization rate indicators with measures of other contexts of care (cost, 
clinical management, satisfaction/experience) that are typically computed at a local level. 



Introduction 

The assessment of the quality of health care has been a problematic 
issue for decades and in all settings, 1,2 including primary care. 3-5 
Starting from the 1970s, 6 a number of quality indicators for primary 
health care has been proposed, 7 but their validity has often remained 
questionable. 3 ' 8,9 An important progress was achieved in the early 
1990s with the concept of 'potentially avoidable' or 'ambulatory 
care sensitive' hospitalizations, 10 which were extensively used to 
indirectly evaluate the performance of primary health care in 
several countries. 11-16 

Based on the assumption that the hospitalization for several 
chronic and acute conditions can be prevented with timely and 
appropriate ambulatory care, 17 the Agency for Healthcare Research 
and Quality (AHRQ) developed a set of 14 primary care quality 
measures, named 'prevention quality indicators' (PQIs), which 
were first released in 2000 18 and which have been validated in the 
USA to compare the performance of national 19-28 and local 
(counties) 29-31 providers. 

Some of the PQIs were also used in Italy to compare the quality of 
primary health care across the regions. 32-35 However, in the Italian 
National Health System (NHS), primary care services are not 
directly managed by the regions, but they are in charge of both 



local health authorities (LHAs) and smaller entities within each 
LHA named health districts (counting ~50 000-60 000 inhabitants 
on average). 36 Thus, regional-level comparison may provide limited 
information. 

The present analysis of the hospital admissions of five Italian 
regions for the years 2008-10 was aimed at evaluating for the first 
time whether PQIs may be used to compare the quality of primary 
health care of smaller territorial entities such as LHA and health 
districts within the Italian NHS. 

Methods 

The list of all PQIs is reported in box 1. We computed all 
composite and 13 of the 14 individual PQIs: PQI 9 (low birth 
weight rate) was excluded because we only obtained data of 
the admissions of people aged >18 years. To compute PQIs, 
we followed strictly AHRQ technical specifications, 37 and all 
details have been reported at the end of the Supplementary 
Appendix. 

In brief, with the exception of PQI 2, which is a percentage, each 
PQI is a ratio between the number of hospital admissions for specific 
disease, and the total number of residents, aged >18 years, in a 
metro area or county. 17 For our analyses, each LHA (the Italian 
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ASL) was a separate metro area, and every PQI was computed for 
each LHA. Thus, each PQI was the rate of admission for a specific 
disease of all the residents of a single LHA (x 100 000). 

Hospital discharge abstracts were officially provided by the 
General Directorate for Health Planning of the Italian Ministry of 
Health, which granted the permission to extract the data of the 44 
LHA of four Italian regions (Abruzzo, Emilia-Romagna, Lazio and 
Lombardia) and the city of Trieste, for the triennium 2008-10. 

The number of residents of each LHA, with age and gender 
strata, was extracted both from publicly available data sets of the 
National Institute of Statistics (Istat) 38 and from other demographic 
data sets provided by the Italian Ministry of Health. For each PQI, 
once the crude admission rates were obtained for each PQI, they 
were age-standardized using an indirect method and 10-year age 



classes. We also computed the standardized admission rates for 
the three composite PQI measures PQI 90, PQI 91 and PQI 92. 39 
PQI 90 represents the overall measure of quality, and it is the sum of 
the numerators of all the individual indicators (excluding PQIs 2, 13 
and 14). PQI 91 and PQI 92 are the sum of the admission rates for 
acute and chronic conditions, respectively. 

We computed the PQIs, as described above, also for each health 
district (as a metro area) to evaluate whether at least some of the 
indicators had enough statistical power to be used for comparing 
smaller territorial entities. Unfortunately, we were able to retrieve 
the information on patient's health district only for a few LHAs of 
the Abruzzo region (n = 7) and the city of Trieste (n = 4) because 
Italian hospital discharge abstracts do not contain such information, 
which must be obtained, with permission, from single LHAs. 

The computation of age- and sex-standardized PQIs by LHA was 
demanding: for each LHA (n = 44), in each year (n = 3), demo- 
graphic and admission data need to be extracted and tabulated in 
several strata (two genders and seven 10-year age classes) for each 
PQI (n = 16). Thus, the overall number of cells to deal with is 29 568 
(44*3* 16*2*7 cells). This is even more complex when the unit of the 
analysis is the health district because there are several districts in 
each LHA. In our analysis, we initially observed no influence of 
gender on the estimates of a pilot analysis on five LHAs from all 
regions, and thus we did not standardize by gender. 

As a secondary analysis, we investigated to what extent composite 
PQIs were correlated with the overall admission rates (once excluded 
Ambulatory Care Sensitive Conditions (ACSCs)), that is, whether 
PQIs did represent a meaningful measure or we could simply use the 
overall admission rate as a proxy of the efficacy/efficiency of the 
primary health care services. Besides an initial univariate analysis 
based on Spearman correlation coefficient, we fitted three 
random- effects linear regression models, in which the dependent 
variables were the three composite PQIs (PQI 90, 91 and 92) in 
the triennium 2008-10. For all models, the cluster units 
were single LHAs (health districts were too few to allow 
meaningful multivariate analyses) and the independent variables 
were overall LHA hospitalization rate (excluding ACSCs), LHA 
total population, year and region. Because of the hierarchical 
structure of our data, in which regions, LHA and years represent 
different levels with potential intra-cluster correlations, 40 the 
analysis was repeated using a random- effects multilevel model 
with three levels of cluster (year, region and LHA). Because the 



Box 1 List of the PQI measures, both individual and composite 


Individual measures; 




PQI 1 — Diabetes short-term complications; 




PQI 2 — Perforated appendix; 




PQI 3 — Diabetes long-term complications; 




PQI 5 — Chronic obstructive pulmonary disease or 


Asthma 


in older adults; 




PQI 7 — Hypertension; 




PQI 8 — Congestive heart failure; 




PQI 10— Dehydration; 




PQI 11 — Bacterial pneumonia; 




PQI 12 — Urinary tract infection; 




PQI 13 — Angina without procedure; 




PQI 14 — Uncontrolled diabetes; 




PQI 15 — Asthma in younger adults; 




PQI 16 — Rate of lower- extremity amputation 


among 


diabetics. 




Composite measures: 




PQI 90— Overall PQI composite (PQIs 1, 3, 5, 7, 8 


, 10, 11, 


12, 15, 16) 




PQI 91— Acute PQI composite (PQIs 10, 11, 12) 




PQI 92— Chronic PQI composite (PQIs 1, 3, 5, 7, 8 


, 15, 16) 


PQI 'Trinity (PQIs 5, 8, 11) 





Table 1 PQIs: hospital admission rates in Italy (2008-10) and the USA (2008-09) 



Indicators (admission rates x 100000) 


Italy 3 






A % 


USA 


Comparison 


USA 


Comparison 














USA vs. Italy, % 




USA vs. Italy, % 




2008 


2009 


2010 


2008-10 


2008 


2008 


2009 


2009 


PQI 1 — Diabetes short-term complications 


12 


10 


10 


-16.7 


62 


+416.7 


62 


+520.0 


PQI 2— Perforated appendix (%) 


29 


30 


31 


+8.1 


28 


-3.4 


29 


-3.3 


PQI 3 — Diabetes long-term complications 


76 


57 


69 


-9.2 


129 


+69.7 


118 


+ 107.0 


PQI 5 — Chronic obstructive pulmonary disease or 


188 


159 


154 


-18.1 


578 


+207.4 


559 


+251.6 


asthma in older adults 


















PQI 7 — Hypertension 


36 


39 


46 


+27.8 


62 


+72.2 


63 


+61.5 


PQI 8 — Congestive heart failure 


336 


301 


399 


+18.8 


400 


+19.0 


381 


+26.6 


PQI 10— Dehydration 


27 


23 


28 


+3.7 


176 


+551.9 


139 


+504.3 


PQI 11 — Bacterial pneumonia 


181 


183 


205 


+13.3 


362 


+ 100.0 


336 


+83.6 


PQI 12 — Urinary tract infection 


59 


59 


65 


+10.2 


206 


+249.2 


197 


+233.9 


PQI 13 — Angina without procedures 


114 


98 


104 


-8.8 


25 


-78.1 


23 


-76.5 


PQI 14— Uncontrolled diabetes 


51 


41 


45 


-11.8 


23 


-54.9 


22 


-46.3 


PQI 15 — Asthma in younger adults 


48 


30 


41 


-14.6 


60 


+25.0 


63 


+110.0 


PQI 16 — Rate of lower-extremity amputation 


7 


7 


7 


0.0 


18 


+ 157.1 


17 


+ 142.9 


among diabetics 


















PQI 90— Overall PQI composite 


1012 


889 


988 


-2.4 


1825 


+80.3 


1714 


+92.8 


PQI 91— Acute PQI composite 


250 


268 


282 


+12.2 


744 


+ 197.6 


672 


+ 150.7 


PQI 92— Chronic PQI composite 


761 


640 


705 


-7.4 


1081 


+42.0 


1042 


+62.8 



a: Abruzzo, Lazio, Emilia-Romagna, Lombardy and the city of Trieste. 
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results for the remaining covariates (hospitalization rate and total 
population) were similar, only the results of the simpler regression 
model were shown to avoid redundancy. 

All analyses were carried out using Stata software, version 11 
(Stata Corporation, College Station, Texas, USA, 2007). The study 
protocol was approved by the ethics committee of the Sapienza 
University of Rome (prot. n. 657/12, 19 July 2012). 

Results 

Study sample and comparison among regions and 
the USA 

We applied the algorithm to compute PQIs to all the 1 1 470 722 
hospitalizations that took place in the five selected regions 
(including ~18 million residents), between 2008 and 2010 
(Supplementary table SI). Overall, the prevalence of admissions 
for ACSC according to AHRQ criteria 17 was 5.7% (n = 648 180), 
ranging from 4.9 to 9.7% across regions and years. 

The hospitalization rates for composite PQIs have been reported 
in table 1 (overall), Supplementary table S2 (stratified by region), 
figure 1 and Supplementary figures SI and S2 (stratified by LHA) 
and figure 2 and Supplementary figures S3 and S4 (stratified by 
district). The overall hospitalization rates for ACSC (composite 
PQI 90) were consistently different across regions during the study 
period, with Abruzzo and Trieste showing rates that were 15-25% 
higher, on average, than those from Lazio and Lombardy 
(Supplementary table S2). 

From 2008 to 2010, the overall hospitalization rates for acute (PQI 
91) and chronic (PQI 92) conditions showed opposite trends: in all 
areas, PQI 91 increased (+12.2% overall; ranging from +3.9% in 
Trieste — in which the rate was already high — to +15.5% in Lazio) 
while PQI 92 decreased (—7.4% overall; ranging from —4.3% in 
Emilia-Romagna to —16.5% in Trieste — Supplementary table S2). 

When compared with the USA (table 1), the rates of PQIs of the 
included Italian regions were generally lower or extremely lower, 
with the exception of the admissions for angina without 
procedures (PQI 13) and uncontrolled diabetes (PQI 14), which 
were higher than in the USA. In the years 2008 and 2009 (last 
available for the USA), the overall rate of potentially preventable 
hospitalizations (composite PQI 90) of the USA was 80.3 and 
92.8% higher than Italian regions, respectively (table 1). 

Use of PQIs to compare local-level providers 

The overall hospitalization rates for ACSC (composite PQI 90) for 
each of the 44 included LHAs, in each year, have been reported in 
figure 1. LHAs showed relevant (and significant) differences among 
each other: the rate between the highest and lowest LHA value each 
year ranged from 2.20 to 2.33. A similar scenario was observed for 
PQIs 91 (acute conditions only) and 92 (chronic conditions only) — 
Supplementary figures SI and S2, respectively. Somewhat smaller 
differences in PQI 90 values were observed across health districts 
(figure 2), although the 'best/worst' admission rate never fell below 
1.97 in the triennium. Again, a comparable picture was obtained 
when analysing PQIs 91 and 92 (Supplementary figures S3 and S4, 
respectively). 

With regard to stability over years, which would be presumable 
given the complex multidisciplinary nature of PQIs and the time 
needed to observe the effects of interventions on health services, 
most of the LHAs and health districts showed little variation in 
composite PQIs' values over the triennium (figures 1 and 2 and 
Supplementary figures S1-S4). Moreover, the rank of most LHAs 
(69.8-76.7%) or districts (83.8-100%) showed small variation across 
the 3 years (Supplementary table S3), indicating that the ranking of 
the performance of LHAs or health districts based on composite 
PQIs is generally stable. 
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Figure 2 PQIs: PQI 90 — overall hospitalization rates for ambulatory care sensitive conditions in 11 health districts from three Italian LHAs 
(LHA of Pescara: Pe01-Pe06; LHA of Lanciano-Vasto-Chieti: district of Francavilla; LHA of Trieste: Tr01-Tr04); years 2008-10 

Table 2 Potential predictors of the rate of hospitalization for an ambulatory care sensitive condition (PQIs 90, 91 and 92) in the years 
2008-10 



Variables 


PQI 90 (overall) 


PQI 91 (acute) 


PQI 92 (chronic) 




Coeff. 


(P) 


Coeff. 


(P) 


Coeff. (P) 


Rate of all hospitalizations excluding PQI conditions (10 x 1000), 10-unit increase 
Total population, 1000-resident increase 
Year, 1-year increase 


0.27 
-0.10 
10.1 


(0.020) 

(0.2) 

(0.3) 


0.16 
-0.08 
27.3 


(0.036) 
(0.051) 
(0.001) 


0.15 (0.029) 
-0.02 (0.8) 
-15.0 (0.011) 



Coeff. = regression coefficient. 

Random-effects linear regression with one level of cluster (LHA) and 124 observations. The analysis was repeated using a random-effects 
multilevel model with three levels of cluster (year, region and LHA), with similar results for the remaining covariates (hospitalization rate 
and total population). 
The unit of analysis was the LHA. 



Adjusting for year, total LHA population and region, all composite 
PQIs were significantly and positively associated with the rate of all 
admissions excluding ACSCs (table 2). In specific, the global hospi- 
talization rate for ACSCs (PQI 90) increased by 0.27 admis- 
sions x 100.000 inhabitants every 10 general hospitalizations more, 
suggesting that the two rates are correlated but PQIs are also 
appraising another construct in addition to health care use. 

With purely descriptive purposes, the rates of all individual PQIs 
of each of the 44 included LHAs and each of the 1 1 included health 
districts, in each year, have been reported in the Supplementary 
tables S4 and S5, respectively. 

Discussion 

With this reanalysis of > 1 1 million hospital admissions in 3 years, we 
evaluated whether PQIs might be used among the measures to 
compare the health care quality of local-level providers within a 
European country characterized by universal coverage such as 
Italy, in which primary health care management is in charge 
of LHAs and health districts. The results of the present study 
support the use of the indicators proposed by AHRQ: composite 
PQIs showed a good discriminative ability among LHAs and among 
districts, a high level of stability during the triennium, and although 
they were significantly correlated with the overall admission rates, 
there was evidence that the rates of preventable hospitalization are 
not simply a reflection of the general hospitalization rates. 
Interestingly, while the admission rates for preventable conditions 
declined by 2.4% during the study period (—7.4% among chronic 
conditions and +12.2% among acute conditions), the rates for non- 
preventable conditions (all the others 41 ) declined by 7.3%, which 
does not seem to indicate an impact on primary health care — at 
least in the short-term and especially for acute conditions — of the 



concomitant in-depth Italian health care system restructuring 42 
(which were only recently focused on primary care organization 43 ). 

PQIs have already been validated to assess primary health care 
quality in several studies carried out in the USA 19-29,44 and other 
countries, 45 including Italy. 32-35 The comparisons, however, were 
carried out mostly at a national 19-25,44 or regional level 34,35 or to 
identify potential predictors of poorer access to primary health 
care. 26-28,32,33 Some technical AHRQ reports reported data at a 
county level, but no attempt at assessing the validity of PQIs was 
made. 30,31 In Italy, an agency of the Minister of Health (Age.Na.s) 
computed PQIs for each province, but LHAs do not always 
correspond to provinces, and data are not publicly available. 46 
In fact, to date, only one study assessed whether PQIs were appro- 
priate to compare small geographical areas, particularly counties 
from Kentucky in the years 2006-08. 29 The results of such analysis 
were in line with our findings: PQIs showed significant and clinically 
important variations across counties, but each indicator had stable 
rates over time within a county. The authors concluded that PQIs 
are potentially suitable for the implementation of a structured 
system for primary health care quality assessment. Having the 
chance to compare the performance of small territorial entities 
might be crucial to establish a meaningful (and acceptable) system 
for the evaluation of the quality of primary care. In several countries 
including Italy, the organizational (and often even the strategic) 
management of primary care services is not set only at a regional 
level, and most responsibility is given to local health care providers. 
Thus, the detection of poor performances and the development of 
interventions aimed at improving the quality of primary care should 
necessarily involve also local-level providers. 

The construct validity of PQIs can be reasonably assumed: besides 
AHRQ, 17 the relationship between ACSCs and other primary health 
care quality was also supported by a recent systematic review 
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managed by the regions, but is in charge of smaller territorial 
entities. 

• For a primary care quality assessment system, it is crucial to 
achieve a high level of precision, to discriminate the per- 
formance of the entities that are responsible for the perform- 
ance itself, and to align hospitalization rate indicators with 
measures of other contexts of care (cost, clinical 
management, satisfaction/experience) that are typically 
computed at a local level. 

• Composite PQIs were able to differentiate local-level pro- 
viders such as Italian local health authorities or health 
districts, which showed small variation in the performance 
ranking across the years, as it might be expected given that 
time is needed to observe the effects of changes on health 
services. 

• In nations with universal health care coverage, composite 
PQIs appeared a valid set of indicators to evaluate the per- 
formance of relatively small primary health care providers. 



including 51 studies, in which the authors concluded that 'most 
studies confirmed the expected relationship between indicators of 
primary health care and hospitalization for ACSCs, showing lower 
hospitalization rates for ACSCs in areas with greater access to 
primary health care'. 47 If further research will confirm PQIs face 
and predictive validity, the next step might be the addition of 
PQIs to enhance the content validity of the sets of indicators that 
are currently used to assess the performance of primary care 
practices. 48 ' 49 

The major strengths of our analysis include the relatively large 
data set, the inclusion of several regions throughout Italy, the use 
of a 3 -year time span and the adoption of exactly the same AHRQ 
criteria for the computation of PQIs (thus increasing comparability). 
However, this study has some limitations that must be considered in 
interpreting the results. First, as most studies on PQIs, we did not 
consider several factors that are beyond the control of family 
physicians (such as disease prevalence, income level, health care 
access), and that may explain part of the differences in potentially 
avoidable admission rates existing between small areas. 14 Notably, 
however, access to care in Italian NHS is granted for all with limited 
or no additional expense. Also, unfortunately in Italy most of the 
epidemiological estimates are based on hospital discharge abstracts; 
thus, it would have been of limited utility to adjust comparisons of 
hospitalization rates using other (likely related) hospitalization rates. 
A second limitation derives from our ability to compute PQI 
admission rates for a small proportion of health districts only, 
since data were retrieved from 3 LHAs out of 44 involved in the 
study (because of privacy barriers and data collection inefficiencies). 
Thus, further studies including a higher number of health districts 
are strongly recommended and caution is required to interpret the 
findings on health districts. Third, the coding of admissions abstract 
could differ among LHA units and this may induce some bias. 50,51 
Fourth, it must be acknowledged that substantive structural differ- 
ences exist between Italian and the US health care systems, 36 and the 
US PQI rates have been shown for merely descriptive purposes: a 
structured comparison between the countries would require extra 
data and analyses. Finally, we only assessed the relatively few diseases 
that were included in PQIs, and it will certainly be useful in the 
future to add some conditions to the analysis, especially those that 
have been examined and validated in previous studies on the Italian 
setting. 51,52 

Italy, as well as several other countries, still misses a structured 
system to evaluate primary health care quality. For these quality 
assessment systems, it is crucial to achieve a high level of 
precision, to discriminate the performance of the entities that are 
responsible for the performance itself, and to align hospitalization 
rate indicators with measures of other contexts of care (cost, clinical 
management, satisfaction/experience 48 ) that are typically computed 
at a local level. The results of the present study support the use of 
composite PQIs, in nations with universal health care coverage, as a 
valid set of indicators to evaluate the quality of relatively small 
primary health care providers (up to ^50 000 enrollees). 
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Key points 

• Outside the USA, AHRQ Prevention Quality Indicators 
(PQIs) have never been used to compare the quality of 
primary care services at a local level; however, in several 
national health systems, primary care is not directly 
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